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Abstract — In this paper, we study the convergence be- 
havior of distributed iterative algorithms with quantized 
message passing. We first introduce general iterative func- 
tion evaluation algorithms for solving fixed point problems 
distributively. We then analyze the convergence of the 
distributed algorithms, e.g. Jacobi scheme and Gauss- 
Seidel scheme, under the quantized message passing. Based 
on the closed-form convergence performance derived, we 
propose two quantizer designs, namely the time invariant 
convergence-optimal quantizer (TICOQ) and the time vary- 
ing convergence-optimal quantizer (TVCOQ), to minimize 
the effect of the quantization error on the convergence. We 
also study the tradeoff between the convergence error and 
message passing overhead for both TICOQ and TVCOQ. 
As an example, we apply the TICOQ and TVCOQ designs 
to the iterative waterflUing algorithm of MIMO interference 
game. 

I. Introduction 

Distributed algorithm design and analysis is a very 
important topic with important applications in many 
areas such as deterministic network utility maximization 
(NUM) for wireless networks and non-cooperative game. 
For example, in H], ||2l, the authors derived various 
distributed algorithms for a generic deterministic NUM 
problem using the decomposition techniques, which can 
be classified into primal decomposition and dual de- 
composition methods. In (3], the authors investigated a 
distributed power control algorithm for an interference 
channel using non-cooperative game and derived an 
iterative water-filling algorithm to approach the Nash 
equilibrium (NE). The interference game problem was 
extended to iterative waterfilling algorithm for a wide- 
band interference game with time/frequency offset in 
in and an iterative precoder optimization algorithm for 
a MIMO interference game in |j5|, jS). The authors 
established a unified convergence proof of the itera- 
tive water-filling algorithms for the SISO frequency- 
selective interference game and the MIMO interference 
game using a contraction mapping approach. Using this 
framework, the iterative best response update (such as 
the iterative power water-filling as well as the iterative 



precoder design) can be regarded as an iterative function 
evaluations w.rt. a certain contraction mapping and the 
convergence property can be easily established using 
fixed point theory Q, E). In all these examples, the 
iterative function evaluation algorithms involved explicit 
message passing between nodes in the wireless networks 
during the iteration process. Furthermore, these existing 
results have assumed perfect message passing during the 
iterations. 

In practice, explicit message passing during the it- 
erations in the distributed algorithms requires explicit 
signaling in wireless networks. As such, the message 
passing cannot be perfect and in many cases, the mes- 
sages to pass have to be quantized. As a result, it 
is very important and interesting to study about the 
impact of quantized message passing on the convergence 
of the distributed algorithms. Existing studies on the 
distributed algorithms under quantized message passing 
can be classified into two categories, namely the dis- 
tributed quantized average consensus algorithms ||9]- 
0141 as well as the distributed quantized incremental 
subgradient algorithms ifTSl - lfTSll . For the distributed 
quantized average consensus algorithms, existing works 
considered the algorithm convergence performance under 
quantized message passing for uniform quantizer |[9], 
ifTOl . lfT2l - |[T4l and logarithmic quantizer [IJ l with fixed 
quantization rate. In lfT2l . lfT4l . the authors also con- 
sidered quantization interval optimization (for average 
consensus algorithms) based on the uniform fixed-rate 
quantization structure. Similarly, for the second cate- 
gory of quantized incremental subgradient algorithms, 
the authors in ifTSl - lfTSl considered the convergence 
performance of fixed-rate uniform quantization. In this 
paper, we are interested in the convergence behavior of 
distributed iterative algorithms for solving general fixed 
point problems under quantized message passing. The 
above works on quantized message passing cannot be 
applied to our case due to the following reasons. First of 
all, the algorithm dynamics of the existing works (linear 
dynamics for average consensus algorthms and step-size 
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based algorithms for incremental subgradient algorithms) 
are very different from the contraction-based iterative 
algorithms we are interested in (for solving fixed point 
problems). Secondly, the above works have imposed sim- 
plifying constraints of uniform and fixed rate quantizer 
design and it is not known if a more general quantizer de- 
sign or adaptive quantization rate could further improve 
the convergence performance of the iterative algorithms. 
There are a few technical challenges regarding the study 
of convergence behavior in distributed contraction-based 
iterative function evaluations. 

• Convergence Analysis and Performance Trade- 
off under Quantized Message Passing: In the 

literature, convergence of distributed iterative func- 
tion evaluation algorithms under quantized message 
passing has not been considered. The general model 
under quantized message passing and how does the 
quantization error affect the convergence are not 
fully studied. Furthermore, it will also be interesting 
to study the tradeoff between convergence error and 
message passing overhead. 

• Quantizer Design based on tlie Convergence Per- 
formance: Given the convergence analysis results, 
how to optimize the quantizer to minimize the 
effect of the quantization error on the convergence 
is a difficult problem. In general, quantizers are 
designed w.r.t. a certain distortion measure such 
as the mean square error lfT9l . ll20ll . However, it 
is not clear which distortion measure we should 
use to design the quantizer in order to optimize the 
convergence performance of the iterative algorithms 
we considered. Furthermore, the convergence per- 
formance highly depends on the quantizer structure 
as well as the quantization rate, and hence, a low- 
complexity solution to the nonlinear integer quan- 
tizer optimization problem is of great importance. 

In this paper, we shall attempt to shed some lights 
on these questions. We shall first introduce a general 
iterative function evaluation algorithm with distributed 
message passing for solving fixed point problems. We 
shall then analyze the convergence of the distributed 
algorithms, e.g. Jacobi scheme and Gauss-Seidel scheme, 
under the quantized message passing. Based on the 
analysis, we shall propose two rate-adaptive quantizer 
designs, namely the time invariant convergence-optimal 
quantizer (TICOQ) and the time varying convergence- 
optimal quantizer (TVCOQ), to minimize the effect of 
the quantization error on the convergence. We shall also 
develop efficient algorithms to solve the nonlinear integer 
programming problem associated with the quantizer op- 
timization problem. As an illustrative example, we shall 
apply the TICOQ and TVCOQ designs to the iterative 
waterfilling algorithm of the MIMO interference game 

0, El. 



We first list the important notations in this paper in 
table H 



TABLE I 
List OF Important Notations . 



II. Iterative Function Evaluations 

In this section, we shall introduce the basic iterative 
function evaluation algorithm to solve fixed point prob- 
lems as well as its parallel and distributed implemen- 
tations. We shall then review the convergence property 
under perfect message passing in the iteration process. 
We shall also illustrate the application of the framework 
using the MIMO interference game in IS), ||6l as an 
example. 

A. A General Framework of Iterative Function Evalua- 
tion Algorithms 

In algorithm designs of wireless systems, many it- 
erative algorithms can be described as the following 
dynamic update equation Q: 

x(t + l) =T(x(t)) (1) 

where x(t) e M" is the vector of state variables of the 
system at (discrete) time t and T is a mapping from a 
subset X C M" into itself. Such iterative algorithm with 
dynamics described by ([T]i is called the iterative function 
evaluation algorithm, which is widely used to so\\& fixed 
point problems Q, IS). Specifically, any vector x* 6 AT 
satisfying T(x*) = x* is called a fixed point of T. If 
the sequence {x(t)} converges to some x* ^ X and 
T is continuous at x*, then x* is a fixed point of T 
JT). Therefore, the iteration in ([T]i can be viewed as 
an algorithm for finding such a fixed point of T. We 
shall first review a few properties below related to the 
convergence of ([T]i. Specifically, T is called a contraction 
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dimension of vector of state variables 
element index of vector 
number of nodes/blocks 
node index/block index 
total number of iterations 
iteration index 
component quantizer of node k (general) 
system quantizer (general) 
scalar quantizer (SQ) 
vector quantizer (VQ) 
component quantizer of node k (SQ) 
system quantizer (SQ) 
quantization index vector (SQ) 
quantization rate vector (SQ) 
component quantizer of node k (VQ) 
system quantizer (VQ) 
quantization index vector (VQ) 
quantization rate vector (VQ) 
the set of nonnegative real numbers 
the set of nonnegative integers 
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mapping if it satisfies some property, which is defined as 
follows; 

Definition 1 (Contraction Mapping): Let T : A' — >■ 
A' be a mapping from a subset X C R" into itself 
satisfying the following property ||T(x) — T(y)|| < 
a||x — y|| (Vx, y G X), where || ■ || is some norm and 
a e [0, 1) is a constant scalar Then the mapping T is 
called a contraction mapping and the scalar a is called 
the modulus of T. ■ 

Remark 1: (Comparison with Step-size Based Incre- 
mental Subgradient Algorithms) The incremental subgra- 
dient algorithms ||2TI can be described as x(t + 1) = 
x{t) — etg(x{t)), where {et} is the step-size sequence 
and g(x{t)) is a subgradient of the objective function 
at x(t) in a minimization problem. Such step-size based 
update algorithms and their associated convergence dy- 
namics are quite different from the iterative function 
evaluation algorithm we considered in ([Til. ■ 

If T is a contraction mapping, then the iterative update 
in ([T]i is called contracting iteration. The convergence of 
([T]i is summarized as follows (Proof can be found in Q): 

Theorem 1 (Convergence of Contracting Iterations): 
Suppose that T : A' ^ A' is a contraction mapping 
with modulus a G [0,1) and that X C M" is closed. 
We have; 

(1) (Existence and Uniqueness of Fixed Points) The 
mapping T has a unique fixed point x* G A'. 

(2) (Geometric Convergence) For any initial vector 
x(0) G X, the sequence {x(i)} generated by ([T]i con- 
verges to X* geometrically. In particular, ||x(t) — x*|| < 
a*||x(0) -x*|| Vi>0. ■ 

In the above discussion, || • || can be any well-defined 
norm. There are many useful norms in the literature. 
However, the commonly used norms can be classified 
into two groups, namely weighted maximum norm and 
Lp norm (1 < p < oo). They are elaborated below; 

• Weighted maximum norm; 



(a™ > 0) 



(2) 



Note that for am = 1 Vm, this reduces to the 
maximum norm, which can also be obtained from 
the Lp norm by taking the limit p ^ oo. 
Lp norm (1 < p < oo); 
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(3) 



Note that for p = 1 we get the taxicab norm and 
for p = 2 we, get the Euclidean norm. 

B. Parallel and Distributed Implementation of Contract- 
ing Iterations 

In practice, large scale computation always involves 
a number of processors or communication nodes jointly 



executing a computational task. As a result, parallel and 
distributed implementation is of prime importance. Infor- 
mation acquisition and control are within geographically 
distributed nodes, in which distributed computation is 
more preferable. In this part, we shall discuss the efficient 
parallel and distributed computation of the contracting 
iteration in (|T|). 

To perform efficient parallel and distributed implemen- 
tations with K processors, the set X is partitioned into 
a Cartesian product of lower dimensional sets, based 
on the computational complexity consideration or the 
local information extraction and control requirement. 
Mathematically, it can be expressed as A' = JlfcLi '^^^ 
where Xk ^ K"'' and X^fcLi "^fe — ^- Let no = and 
Mk - {m G N : ^ti '^i-i + 1 < m < ^^i ^i} be 
the index set of the A;-th component set Xk (^ <k < K), 



where N is the set of integers. Thus, X^ = W 
where Xm ^ M^. Any vector x G A' is decomposed 
as X = (xi,-- - ,Xif) with the fc-th block component 

Xfc = (Xm)m.<^Mk & Xk = HmeMk '^'^ 

mapping T : X ^ X 'v& decomposed as T(x) — 
(Ti(x),-- - ,T/f(x)) with the A;-th block component 
Tfe : AT ^ Xk. 

When the set AT is a Cartesian product of lower dimen- 
sional sets Xk, block-parallelization with K processors 
can be implemented by assigning one processor to update 
a different block component. The most common updating 
strategies for xi , • • • , x/f based on the block mapping T 
are; 



Jacobi Scheme; All block components xi, ■ 
are updated simultaneously, i.e. 



Xfe(i + l) = Tfe(x(<)), \<k<K (4) 

• Gauss-Seidel Scheme; All block components 
xi , • • • , x/^ are updated sequentially, one after the 
other, i.e. 



xfc(t+l) = Sfe(x(t)), l<fc<A' 

where S/; : AT — > A'^ given by 

S/c(x) 
Tfe(x), 

Tfe(Sl(x),-- - ,Sfe_l(x),XA;, • • • ,XA'): 



(5) 



(6) 

k = 1 

2<k<K 



is the k-i\\ block component of the Guass- 
Seidel mapping S : A' — > A', i.e. S(x) = 
(Si(x),... ,S^f(x)). 
Both Jacobi Scheme and Gauss-Seidel Scheme belong 
to synchronous update scheme^ Specifically, Jacobi 
Scheme assumes the network is synchronized, while the 

'Due to page limit, we shall illustrate the design for synchronous 
updates in j4) and (sj. However, the scheme can be extended to deal 
with totally asynchronous updates easily (7), which will be further 
illustrated later in footnote |5] 
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Gauss-Seidel Scheme assumes the network provides a 
(HamOtonian) cycHc route Q. 

The general weighted block-maximum norm on M", 
which is usually associated with the block partition of 
the vector x, is defined as ||7J: 

11x11,^,, = max ^ (7) 

k Wk 

where w (wi, • • • , wk ) > is the vector weight and 
II • II is the norm for the fc-th block component x^, which 
can be any given norm on R"*, such as the weighted 
maximum norm and the Lp norm (1 < p < oo) defined in 
(|2|i and (O. The mapping T : ^ — > is called a block- 
contraction with modulus a G [0, 1) if it is a contraction 
under the above induced weighted block-maximum norm 
I ■ 1 1 block ^i'^h modulus a. The convergence of the Jacobi 
scheme and Gauss-Seidel scheme based on the block- 
contraction is summarized in the following theorem Q: 

Theorem 2: {Convergence of Jacobi Scheme and 
Gauss-Seidel Scheme) If T : Af — Af is a block- 
contraction, then the Gauss-Seidel mapping S is also a 
block-contraction with the same modulus as T. Further- 
more, if X is closed, then the sequence {x(t) } generated 
by both the Jocobi scheme in (|4|i and the Gauss-Seidel 
scheme in (|5]l based on the mapping T converges to the 
unique fixed point x* e A' of T geometrically. ■ 

C. Application Example — MIMO Interference Game 

The contracting iteration in ([T]i is very useful in solving 
fixed point problems. Fixed point problem is highly 
related to distributed resource optimization problems in 
wireless systems [31, 131, ||6l, ll22ll . For example, finding 
the Nash Equilibrim (NE) of a game is a fixed point prob- 
lem. In this subsection, we shall illustrate the application 
of contracting iterations using MIMO interference game 
Is), IS] as an example. 

Consider a system with K noncooperative transmitter- 
receiver pairs communicating simultaneously over a 
MIMO channel with N transmit antenna and N receive 
antenna Q, 161. The received signal of the fc-th receiver 
is given by: 

Yk = HfcfcSfe + ^ HjkSj + rife (8) 

where Sfe £ and e are the vector transmitted 
by the fc-th transmitter and the vector received by the fc-th 
receiver respectively, Hfefe £ i£_nxn jj^g direct-channel 
of the fc-th link, Hjk E C^^^ is the cross-channel from 
the j-th transmitter to the fc-th receiver, and rife G is a 
zero-mean circularly symmetric complex Gaussian noise 

^ Since in general, the norm of each block component may not be 
the same, subscript k is used in || ■ 



vector with covariance matrix Rn^. . For each transmitter 
fc, the total average transmit power is given by 

E[||sfe||2] =Tr(Pfe) <Pfe (9) 

where Tr(-) denotes the trace operator, Pfe = ElsfeS^^^] 
is the covariance matrix of the transmitted vector Sfe 
and Pfe is the maximum average transmitted power The 
maximum throughput of link fc for a given set of users' 
covariance matrices Pi, • • ■ , Pk is as follows 

rfe(Pfe, P_fe) = logdct (I + HffcR:i(P_fe)HfefePfe) 

(10) 

where R-fe(P„fe) ^ Rn, + Ej^k^ikPjtifk is the 
noise covariance matrix plus the MUI observed by user 
fc, and P_fe = (Pj)j^fe is the covariance matrix of all 
other users except user fc. 

In the MIMO interference game Q, 161, each player fc 
competes against the others by choosing his transmit co- 
variance matrix Pfe (i.e., his strategy) that maximizes his 
own maximum throughput rfc(Pfe, P_fc) in ( fTOb , subject 
to the transmit power constraint in the mathematical 
structure of which is as follows 

(g)nmx rfe(Pfe,P_fc) Vfc (11) 

s.t. Pfe G ^fe 

where ^fe 4 |p,^ e C^>^n . ^ 0,Tr(Pfe) = Pfe} 
is the admissible strategy set of user fc, and Pfe )== 
denotes that Pfe is a positive semidefinite matrix. 
Given fc and P_fe G ^^k, the solution to the non- 
cooperative game ( fTTT i is the well-know waterfiUing solu- 
tion P^ = Wrfe(P_fe), where the waterfiUing operator 
WFfe(P_fe) can be equivalently written as 121 

WFfe(P_fe) = [ - (Hf,R:i(P_fe)Hfefe)-']^^ (12) 

where [Xq] ^ — argminze.^:' ||Z — Xo||f denotes the 
matrix projection of Xq w.rt Frobenius nor p onto 

the set The NE of the MIMO Gaussian interference 
game is the fixed point solution of the waterfiUing map- 
ping WF : -> ^, i.e. P^ = WFfe(PlJ Vfc, where 
^ ^ ^1 X • • ■ X and WF = (WFi, • ■ • , WFa')- 
In 0, it is shown that under some mild condition, the 
mapping WF is a block-contractior0 w.rt. || • |||^biock- 
Therefore, the NE can be achieved by the following 
contracting iteration 

P(t+l) = WF(P(t)) (13) 

^If we arrange AIN elements of a M X N matrix X as a 
AfAf -dimensional vector x, then the Frobenius norm of matrix X is 
equivalent to the L2 norm of vector x. 

After rearranging the elements of the N X N covariance matrix 
as a -dimensional vector, the block-contraction WF w.rt. |M|^(,|o(.|j 
is equivalent to a block-contraction w.rt. || ■ defined in (7) with 

each II ■ life being L2 norm. 
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where P = (Pi, • ■ • ,Pk)- It can be easily seen that the 
waterfilHng algorithm for the MIMO interference game 
in (T3[ is a special case of the contracting iterations in 
([T]i- In our general model, x in ([T]i corresponds to P in 
( fT3] i; the block-contraction mapping T in ([T]i corresponds 
to WF in ( fT3] i: the fc-th block component corresponds 
to the covariance matrix P^; the fc-th block component 
mapping T/j corresponds to WFfc. 

For the parallel and distributed implementation, we 
can partition the variable space ^ = Ylk where 
each variable space ^k corresponds to the covariance 
matrix of the fc-th link. In each iteration, the receiver 
of each hnk k needs to locally measure the PSD of the 
interference received from the transmitter of the other 
links, i.e. X^j^^fc Hj'cPj(i)Hj|, computes the covariance 
matrix of the fc-th link and transmits the computational 
results to the associated transmitter. There are two dis- 
tributed iterative waterfilling algorithms (IWFA) based 
on this waterfilling block-contraction, namely simultane- 
ous IWFA and sequential IWFA, which are described as 
follows: 

• Simultaneous IWFA; It is an example of the Jacobi 
scheme, which is given by 

Pfe(t+1) = WF(P_fe(t)), l<k<K 

• Sequential IWFA: It is an example of the Gauss- 
Seidel scheme, which is given by 

P WF(P_,(i)), if(i + l)modA- = fc 

fk[T+^) - I p^^^^^ otherwise 

III. Contracting Iterations under Quantized 
Message Passing 

In this section, we shall study the impact of the quan- 
tized message passing on the contracting iterations. We 
shall first introduce a general quantized message passing 
model, followed by some general results regarding the 
convergence behavior under quantized message passing. 



We assume there are K processing nodes geographi- 
cally distributed in the wireless systems. Fig. [T] illustrates 
an example of ii'-pair MIMO interference game with 
quantized message passing. The system quantizer can 
be characterized by the tuple Q = {Qi, Qk), where 
Qk is the component quantizer (can be scalar or vector 
quantizer) for the fc-th node. Qk can be further denoted 
by the tuple Qk = (ffc, £k ■ Xk ^ Ik is an encoder 
and T>k : Ik is a decoder. Ik = {1, • • • , 2^'=} 

and Lk are the index set and the quantization rate of the 
component quantizer Qk- Xk is the reproduction code- 
hook, which is the set of all possible quantized outputs 
of Qk EH. The quantization rule is completely specified 
by Qk ■ Xk Xk- Specifically, the quantized value is 
given by Xfc = Qfc(xfc) = Vk{£k{'^k))- Each node fc up- 
dates the fc-th block component of the n-dimensional 
vector X, i.e. computes Xfc(t + 1) = Tfc(x(t)). The 
encoder £k of Qk accepts the input Tfc(x(t)) and 
produces a quantization index /fe(t) = fA:(Tfc(x(i))). 
Each node fc broadcasts the quantization index /fe(i). In 
other words, the message passing involves only the quan- 
tization indices I(t) = {lk{t), ■ • ■ , Ixit)) instead of the 
actual controls T(x(t)) = (Ti (x(t)) , • • • ,TK(x(t))). 
Upon receiving the quantization index Ik{t), the decoder 
2?fc of Qk produces a quantized value Xk{t + 1) = 
tfc(x(t)) = Vk{h{t)) = Tfc(x(i)) +efc(i). Therefore, 
the contracting iteration update dynamics of (HJ with 
quantized message passing can be modified as: 



A. General Model of Quantized Message Passing 
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Fig. 1. Illustration of X-pair MIMO interference game. 



x(t + l) = T(x(t)) +e(t) 



(14) 



where e{t) G R" is the quantization error vector at time 
t- The quantizer design affects the convergence property 
of the iterative update algorithm fundamentally via the 
quantization error random process e{t)- Generally, the 
update of each block component is based on the latest 
overall vector, because : X Xk- Thus, the 
decoders of the system quantizer X> = (Vi,--- ,2?^) 
is needed at each node. On the other hand, the fc-th 
node only requires the encoder £k of the corresponding 
quantizer component Qk- 

Consider the application example in Section lTl-Cl under 
quantized message passing. The system quantizer Q = 
(Qi, Qk) can be applied in the MIMO interference 
game with K noncooperative transmitter-receiver pairs 
as illustrated in Fig. [T] Specifically, for the fc-th link, the 
encoder £k is placed at receiver and the decoder Vk is 
placed at the transmitter The MIMO interference game 
under quantized message passing will be illustrated in 
the following example: 

Example 1: (MIMO Interference under Quantized 
Message Passing) In the t-th iteration, the receiver of 
the fc-th link locally measures PSD of the interfer- 
ence received from the transmitter of the other links, 
i-e- Y.j^k HjfcPj(t)Hj|, and computes WFfe(P_fe(^)). 
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The encoder £k of Qk at the fc-th receiver en- 
codes WFfc(P_fc(t)) and passes the quantization index 
^ £:fc(WFfe(P_fe(t))) to the A:-th transmitter. The 
decoder Vk of Qk at fc-th transmitter produces a quan- 
tized value Pfe (i + 1) = Pfc {Ik (t)) = WFfe (P_fe {t)) + 
ek{t). The contracting iterative update dynamics of (fT3] l 
for the MIMO interference game under quantized mes- 
sage passing is given by: 



P(t+1) =WF(P(i)) +e(<) 



(15) 



B. Convergence Property under Quantized Message 
Passing 

Under the quantized message passing, the convergence 
of the contracting iterations is summarized in the follow- 
ing lemma: 

Lemma 1: {Convergence of Contracting Iterations un- 
der Quantized Message Passing) Suppose that T : AT — s- 
Af is a contraction mapping with modulus a G [0, 1) and 
fixed point x* G X, and that X C M" is closed. For any 
initial vector x(0) G X, the sequence {x(t)} generated 
by ( fT4l i satisfies: 

(a) ||x(t) - x*|| < aix(O) - x*|| + E{t) Vt > 1, 
where E{t) = a*^^ X][=o "^"^ 11^(011 accumulated 
error up to the time t induced by the quantized message 
passing. 

(b) For each t, if there exists a vector et G M" such 
that ||e(t)|l < ||ef||, then E{t) < E{t), where E{t) = 

(c) If ||ei|| = •■• = ||et|| = ||e||, then E{t) < E{t), 
where E{t) = T^l|e|| with hmiting error bound 
E{oo) ^ Mmt^ao E{t) = YT^- Furthermore, define the 
stationary set as § = {Q(x) : ||x - x*|| < E{oo)}. The 
sufficient condition for convergence is x = Q(T(x)) 
Vx G § and the necessary condition for convergence is 
3x G §, such that x = Q(T(x)). 

Proof: Please refer to Appendix A for the proof. ■ 
Note that, in the above lemma, the norm j| • || can be any 
general norm. In the following, we shall focus on char- 
acterizing the convergence behavior of the distributed 
Jacobi and Gauss-Seidel schemes under quantized mes- 
sage passing with the underlying contraction mapping 
T defined w.r.t. the weighted block-maximum norm 
II ■ 1 1 Hock S-Q. Under quantized message passing, the 
algorithm dynamics of the two commonly used parallel 
and distributed schemes can be described as follows: 

. Jacobi Scheme under Quantized Message Pass- 
ing: 

Xfc(i + 1) = Tfc(x(i)) + efc(t), l<k< K (16) 

. Gauss-Seidel Sclieme under Quantized Message 
Passing: 

Xfc(t + 1) = Sfc(x(t)) +efe(i), l<k<K (17) 



where 
S,(x) 

T4Si(x)+ei,-- - , 

Sfe_i(x) + efe_i,Xfe, 



(18) 



k = 1 



2<k< K 



Applying the results of Lemma [U the convergence 
property of the distributed Jacobi and Gauss-Seidel 
schemes in ( fTSI ) and ( fTTI i can be summarized in the 
following lemma. 

Lemma 2: {Convergence of lacobi Scheme and 
Gauss-Seidel Scheme under Quantized Message Passing) 
Suppose that T : A' ^ AT is a block-contraction map- 
ping w.r.t. the weighted block-maximum norm || • W^^^^ 
with modulus a G [0,1) and fixed point x* G X, and that 
X C M" is closed. For every initial vector x(0) G X, 
the sequence {x(t)} generated by both the Jacobi scheme 
and the Gauss-Seidel scheme under quantized message 
passing in ( fT6b and ( [TtI i satisfied 

(a) ||x(t) - xllS'^,, < a*||x(0) - x*|L^„,, + S^ToekW 
> 1, where E^At) = V*J a-'||e(/)r„.v 



for Jacobi 



^block(*) 

scheme 



1-a 

scheme 



and i?, 

T.t>-'Mi)\\Z^^ 



block (0 



for Gauss-Seidel 



(b) If condition in (b) of Lemma [T] holds 
, then £;bTock(0 < 4Tock(i), where 



w-r.t. II • 

E 



-biock(*) = a* ^ Ez=o " 
and 4Tock(i) = 
Gauss-Seidel scheme. 



block 



for Jacobi scheme 



^*"'Elo""'l|edlbTock for 



(c) If condition in (c) of Lemma [T] holds w.rt. 
• llbTock^ then i?bToek(0 < ^bTock(i)^ where E^,^^^{t) ^ 



l-g |[p||w 
1-Q ll'^llblock 



for Jacobi scheme with £^^^^,1^(00) 



for 



ri^ll'^llbl'ock ■^Mock(^) - T^TTT^II^IIblock 

Gauss-Seidei scheme with 4Tock(oo) = (1^ ll^llS^ock- 
Furthermore, define the stationary set as §^ ^ 



block 



X — X 



I block 



< £'Mock(oo)}- The sufficient 



{Q(x) 

condition and necessary condition are the same as those 
in Lemma [U 

Proof Please refer to Appendix B for the proof. ■ 

'Our analysis can be extended to totally asynchronous 
scheme in which the results of Lemma |2] becomes: 



^bTock(*) 



1-a 



Et— i 

(1 - a)||x(0) - x-ll-,,, we have £^^,(4) = 



W -"block 



^block(°'^) ~ ll'=llblock- 

Theorem (Proposition 2.1 in Chapter 6 of (7]), we can prove (c) 
(similar to the proof of Theorem 12 in (6]). The proof is omitted here 
due to page limit. Since the error bound result of totally asynchronous 
scheme is similar to Jacobi Scheme and Gauss-Seidel Scheme, our 
quantizer design later can be applied to the asynchronous case. 

''Compared with Jacobi scheme, Gauss-Seidel scheme and totally 
asynchronous scheme have extra error terms ^^^^ and ' respec- 
tively, and > ^7"^^ > 1. 

^ 1 —a 1 —a 



(C) If 
(l-c)^ 



(b) 



and 



, . By the Asynchronous Convergence 
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Remark 2: As a result of Lemma [T] and Lemma |2] the 
effect of quantized message passing affects the conver- 
gence property of the contracting iterative algorithm in a 
fundamental way. From Lemma|2] the Jacobi and Gauss- 
Seidel distributed iterative algorithms may not be able 
to converge precisely to the fixed point under quantized 
message passing due to the term -EwocklO- 

IV. Time Invariant Convergence-Optimal 
Quantizer Design 

In this section, we shall define a Time Invariant 
Quantizer (TIQ) and then formulate the Time Invariant 
Convergence-Optimal Quantizer (TICOQ) design prob- 
lem. We shall consider the TICOQ design for the scalar 
quantizer (SQ) and the vector quantizer (VQ) cases 
separately. Specifically, the component quantizer Qk 
of the fc-th node can be a group of scalar quantizers 
Qk = (2m)meA4fc or a vector quantizer Q^. In the 
SQ case, each element T„i{-) (m G Mk) of the vector 
Tfc(-) is quantized by a coordinate scalar quantizer 
separately. However, in the VQ case, the input to a vector 
quantizer is the vector Tfc(-). 

Definition 2 (Time Invariant Quantizer (TIQ)): 
A Time Invariant Quantizer (TIQ) is a quantizer 
Q — {£, T>) such that E and T) are time invariant 
mappings. ■ 

The system scalar TIQ can be denoted as Q" = 
(Qf, • • • , Q™, • • ■ , Qn)- Let L« = (Lf , . • • , be the 
quantization rate vector for the system scalar TIQ Q", 
where £ Z"*" is the quantization rate (number of bits) 
of the coordinate scalar quantizer Qf„ (1 < < n). 
The sum quantization rate of the system scalar TIQ 
is given by X]m=i ^m- Similarly, the system vector TIQ 
can be denoted as Q" = (Q^', • ■ • , Q^, • • ■ , 0"^). Let 
— (L", • • • , L""^) be the quantization rate vector for 
the system vector TIQ \ where € Z"*" is defined as 
the quantization rate (number of bits) of the coordinate 
vector quantizer Q^! {I < k < K). The sum quantization 
rate of the system vector TIQ Q" is given by '}2^k=i ^k- 

Using Lemma|2](c), the limiting error bound of the al- 
gorithm trajectory is given by E^„^i,{oo) = j4^||e||^„^i^ 

(Jacobi scheme) or E^„^y,{oo) ^ n^l|e|lwock (Gauss- 
Seidel scheme). Therefore, the TICOQ design, which 
minimizes E^o^^^ioo) under the sum quantization rate 
constraint, is equivalent to the following: 
Problem 1 (TICOQ Design Problem): 

(19) 



maxxeA- l|x - Q'{^)\\Z,,^ (SQ case) 



maxxeA- ||x - Q^'(x) 



(VQ case). 



min 


l|e||wock 






71 


s.t. 






m— 1 




K 


or 






k=l 



where ||( 

or llellbTo 

Remark 3 (Interpretation of Problem\l}: Note 
that the optimization variable in Problem [T] 
is the system TIQ or Q" . The objective 

function ||e||^„^k = maxxG^r ||x - Q"(x)||^„^i^ or 
l|e|lwock = maxxeAfllx - Q"(x)||Sfock obviously 
depends on the choice of the system TIQ or 
Q^. Furthermore, the constraint ( |20] | or ( |2TI ) is the 
constraint on the quantization rate L" = {Lf, ■ ■ ■ , LfJ or 
L" = {L\, • • • , i^), which is also an effective constraint 
on the optimization domains of or Q" , respectively. 
It is because Lf^ or is a parameter (corresponding 
to the cardinality of the index set, i.e. = 2^'" or 
= 2^'=) of the encoder and decoder of Qf^ or Q^. 
The Lagrangian function of Problem [T] is given by: 
C^iQ\lin^m\^ioc^ + l^'iEl=iL'm-~L) (SQ case) 

or £"(2",^") = llell^Ioek + l^^iEtiLl - L) (VQ 
case), where /i" or /^i" is the Lagrange multiplier (LM) 
corresponding to the constraint ( l20l ) or (l2Tl i. Hence, 
the optimization problem [T] can also be interpreted 
as optimizing the tradeoff between the convergence 
performance ||e||y and the communication overhead 

E:Li Lfn or Ef=i LI The LM or can be 
regarded as the per-iteration cost sensitivity. 

Remark 4 (Robust Consideration in ProblemU]): 
The optimization objective ||e||yQj,|j in (T% actually 
corresponds to a worst case error in the algorthm 
trajectory. In other words, the TICOQ design is trying 
to find the optimal TIQ which minimizes the worst 
case error In fact, the algorithm trajectory x(t) is a 
random process induced by the uncertainty in the initial 
point x(0). In general, we do not have knowledge on 
the distribution of x(i) due to the uncertainty on x(0). 
Hence, the solution to Problem [T] (optimizing the worst 
case error) offers some robustness w.rt. the choice of 
x(0). ■ 

In the following, we shall discuss the scalar and vector 
TICOQ design based on Problem [T] separately. 

A. Time Invariant Convergence-Optimal Scalar Quan- 
tizer 

We first have a lemma on the structure of the optimiz- 
ing quantizer in the scalar TICOQ design in Problem [T] 

Lemma 3 (Structure of the Scalar TICOQ): If each 
component norm || • ||fc on K"*^ (1 < fc < K) of the 
weighted block-maximum norm || ■ ||yock defined by 
^ is monotone (or absolute) norrrQ, then the optimal 
coordinate scalar quantizer (1 < m < n) w.r.t. the 
worst case error ||e||^„^|^ = maxxe^r ||x - Q'{y^)\\uock 
is a uniform quantizer 

^A vector norm is monotone if and only if it is absolute 1231 . 
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Proof: Please refer to Appendix C for the proof. ■ 
While the optimization variable in Problem [T] (SQ 
case) is = (Qi, • • • , <3^). using Lemma |3] we 
can restrict the optimization domain of each coordi- 
nate scalar quantizer Qf„ (1 < m < rt) to uniform 
quantizer without loss of optimality. Thus, the worst 
case error of the m-th coordinate is given by \em\ — 

I A* I 

where \Xrn\ is the length of the interval (xm 6 Xm), 
and the remaining optimization variable is reduced from 
= {Ql---,Q'J to = (Lf,...,L^). Scalar 
TICOQ design in Problem [T] w.rt. = (if,--' ,if,) 
is a Nonlinear Integer Programming (NLIP) problem, 
which is in general difficult to solve. Verifying the 
optimality of a solution requires enumerating all the 
feasible solutions in most cases. In the following, we 
shall derive the optimal solution to the scalar TICOQ 
design in Problem [T] w.r.t. the weighted block-maximum 
norm j| ■ defined by (|7]i, in which each component 
norm || • \\k is the weighted maximum norm and the Lp 
norm separately. 

Theorem 3 (Solution for Weighted Maximum Norm): 
Given a weighted block-maximum norm || • Hyo^jj defined 
by (|7]i (parameterized by w = {wi,--- ,wk)) with 
I ■ II being the weighted maximum norm || ■ ||^ 
defined by (|2|i (parameterized by a.^ = {am)meMk)' 
Kt = (log2 where C„ ^ .^^^j^^f^^^^^M.^y 

![■] is indicator function and r > is a constant related 
to the LM of the constraint ( [2Q] i chosen to satisfy the 
constraint X]m=i(loS2 %^)^ = L. The optimal integer 
solution of Problem [T] for the SQ case is given 



[if*]!, ifm<E™=i(i^ 
[Lf*.\, otherwise 



Li™ J) 



(22) 

The optimal value of Problem [T] under continuous relax- 
ation is r. 

Proof: Please refer to Appendix D for the Proof. ■ 

Theorem 4 (Solution for Lp Norm): Given a 
weighted block-maximum norm || • W^^^^ defined 
by (|7]i (parameterized by w = [wi,--- ,wk)) 
with II • \\k being the Lp norm || • ||p defined by 
^ (parameterized by p), the optimal solution of 
Problem [T] for the SQ case with continuous relaxation 
(L^ G M+) is Z^* = ilog2(^^7^ , 1 V 1), 

and r are constants related to the LM of the 
constraint (l2Tl i chosen to satisfy the constraint 
Ef=i ErneXfc I log2(^ VI) = L and complementary 
slackness conditions ^{T.mGMS'^rn A Tk) - t) =0 



^We arrange the real sequence zi 
denote them as z^i^ > ■ ■ ■ > ^[m] 



, z„ in decreasing order and 
..>■■•> Z[n]. where 2[„] 
represents the m-th largest term of {zm}- 



(V/j € {1, • • • , The optimal value of Problem [T] 

with continuous relaxation is r?. 

Proof: Please refer to Appendix D for the Proof. ■ 

Remark 5 (Determination of {ri, • • ■ , tk} and t): 
Solving for {ri, - - ,tk} and r involves solving 
a system of K + 1 equations with K + 1 
unknowns. We have 2^ — 1 valid cases for 
the above system of equations according to 
Tk > max,„e7n^. C,„ or Tk < max,„g7n^. C„i (Vfc). 
Firstly, if Tk > nmxmeMk (VA:), then L^^ = 
(Vto), which is not a valid case. Therefore, without 
loss of generality, assume Tk < m&yimeMk C*™ 
V/c e {1,---,A^} and > maxmeMkCm 

yk e {N + 1, - ■ ■ ,K}. The system of A' + 1 
equations and unknowns reduce to iV + 1 equations, 
which are given by: E^eM, ^ log2(^ VI) = L 

and E,„eA^,(C™ A r,.) = r (Vfc G {I,-- - ,7V}), and 

+ 1 unknownQ {^i, • • • ,tn} and r. 

B. Time Invariant Convergence-Optimal Vector Quan- 
tizer 

We first have a lemma on the structure of the optimiz- 
ing quantizer in the vector TICOQ design in Problem 

Lemma 4 (Structure of the Vector TICOQ): If the 
component norm || • ||fc (1 < fc < K) on of the 
weighted block-maximum norm || • W^^^^ defined by (|7| 
is monotone (or absolute) norm, each vector TICOQ 
Q^* is a rifc-dimensional lattice quantizer, the structure 
of which is uniquely determined by the norm || • ||fe 
on I 



In particular, if 



(1 < fc < AT) is L2 



norm, each vector TICOQ Q^* is the thinnest lattice 
for the covering problem in Euclidean space; If || • ||fc 
(1 < fc < K) is weighted maximum norm, each vector 
TICOQ Q1* reduces to Uk coordinate scalar TICOQ 
("^ ^ J^k) with scalar quantizatior03 of each 
coordinate i'm G A^fe) of x^,. 

Proof: Please refer to Appendix E for the proof. ■ 
A lattice is a regular arrangement of points in n- 
dimensional space that includes the origin. "Regular" 
means that each point "sees" the same geometrical en- 
vironment as any other point ||20| . The lattice covering 



V a = maxjx, a} and x A a = minja:, a}. 

'"For example, consider K = 2, n = 4, Mi = {1,2} and 
2 



M2 = {3,4} . We have 2^ - 1 = 3 valid cases. Case 1 ( 
Ti < maxmeMi Cm and t2 < maxmeM2 Cm ) We have 3 
equations: J^Li J2meMk ^ l°g2(%^ V 1) = L, T,meMk (C'" ^ 
Tj.) = T (Vfc G {1,2}), and 3 unknowns: {ri,T2},r. Case 2 ( 
ri < maxmeMi Cm and T2 > max^g^^ Cm ) We have 2 
equations: T,meM^ | l°g2(^ V 1) + = L.E^e^, (C™ A 
ri) = r, and 2 unknowns: ri, r. Case 3 ( ti > maxmeMi Cm 
and T2 < maxmgjvi2 Cm ) We have 2 equations: + 0-1- 
E^eAI. ^I°g2(^ V 1) = L,j:^^^,^{Cm A T2) = r, and 2 
unknowns: T2, t. 

"in other words, the vector TICOQ design reduces to scalar TICOQ 
design discussed in Theorem [3] 
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problem asks for the most economical way to arrange 
the lattice points so that the n-dimensional space can 
be covered with overlapping spheres whose centers are 
the lattice points, i.e. tries to find the thinnest (i.e. 
minimum densitvT^ lattice covering 1241 . The thinnest 
lattice coverings known in all dimension n {n < 23) are 
the dual lattice A* (when 1 < n < 5, A* is known 
to be optimal) ||24|. The worst case error of the dual 
lattice A*^ for the fc-th node when || • ||fc is Lp norm 
(p > 2) is less than that measured in L2 norm (Appendix 
E). Therefore, if || • ||fc is Lp norm (p > 2), we can 
solve the TICOQ design (VQ case) in Problem [T| using 
dual lattice structure At 



Jiven by ||efc||A; = 



n„ 



Thus, the worst case error is 



»fc(»fc+2) t 



/ 1 I V 12(nfc+l) ^ 

(Appendix E), where \Xm\ is the length of the interval 
Xm {xm G '^m) (1 < TO < ??), and the remaining opti- 
mization variable is reduced from Q" — (Q", • • • , Q^-) 
to L" = (L^, • • • , L^). Similarly, vector TICOQ design 
in Problem [U w.r.t. L" = {Ll,---,L]^) is also a 
Nonlinear Integer Programming (NLIP) problem, which 
is in general difficult to solve. In the following, we shall 
derive the optimal solution to the vector TICOQ design 
in Problem [T] based on dual lattice A*^ . 

Theorem 5 (Solution for Dual Lattice quantizer): 
Given a weighted block-maximum norm || • H^ock 
defined by (|7]i (parameterized by w = (wi, • • ■ ,wk)) 
and the dual lattice {A*^ : I < k < K} 
quantizer, let L"^* = n {\og2 ^r)^ , where 



1 



12(nfc + l) ' 



and T > is a 



constant related to the LM of the constraint (ISTT i chosen 
to satisfy the constraint X^aLi "'=(^'-'82 "T^)^ = L. The 
optimal integer solution of Problem [T] for the VQ case 
w.rt. dual lattice {A*^^ : 1 < fc < K} quantizer when 
ni = • • • = riK is given by: 



[Zf^ji, if fc<Ef=i(^r- L^rJ) 

[Lj'^^jJ, otherwise 



L 



[k] 



(23) 



The optimal value of Problem [T] under continuous relax- 
ation is T. 

Proof: Please refer to Appendix E for the Proof. ■ 

C. Tradeoff between Convergence Error and Message 
Passing Overhead 

In this subsection, we shall quantify the tradeoff be- 
tween the convergence error of the algorithm trajectory 
and the message passing overhead using TICOQ. Specif- 
ically, the steady-state convergence error in the algo- 
rithm trajectory is related to E^g^^^{oo) and the message 
passing overhead is related to the sum quantization rate 

'^The density of a covering is the defined as tlie number of splieres 
that contain a point of the space 1241 . 



(number of bits) L of the system quantizer. The following 
lemma summarizes the tradeoff result. 

Lemma 5: {Performance Tradeoff of the Scalar and 
Vector TICOQ) For L>L'{Le Z+), where 

{YZi=i log2 Cjn - nlog2(min,„ C,„), WM norm 
YZi=i log2 Cra~n log2 ( min™ Cm),Lp norm 
Ylk=i log2 Dk- K log2(minfc D^), dual lattice 

(24) 

Cm = (I]fe=i"fcl[TO G Mk])Cm, the limiting error 
bound of the scalar and vector TICOQ considered in this 
section is given by 4Tock(oo) = T^C>(2-^). 

Proof: Please refer to Appendix F for the proof. ■ 
Remark 6: As L -> 00, E^g^^.{oo) — >• 0, which 
reduces to the conventional convergence results with per- 
fect message passing. On the other hand, £'^^^^(00) > 
for finite L and hence, we cannot guarantee the conver- 
gence behavior of the contracting iterations with TICOQ 
to the fixed point x* of the contraction mapping T 
anymore. Nevertheless, the convergence error decreases 
exponentially w.rt. the message passing overhead L. ■ 

V. Time Varying Convergence-Optimal 
Quantizer Design 

Similar to Section [V] we shall define a Time Varying 
Quantizer (TVQ) and then formulate the Time Varying 
Convergence-Optimal Quantizer (TVCOQ) design prob- 
lem. We shall consider the TVCOQ design for both the 
SQ and VQ cases separately. 

Definition 3 (Time Varying Quantizer (TVQ)): 
A Time Varying Quantizer (TVQ) is a quantizer 
Q{t) = {£{t),V{t)) such that 8{t) and V{t) 
changes with time. In other words, the quantizer 
Q(t) = {£{t),V{t)) at the t-th iteration is function of 
time t. ■ 

The system scalar TVQ at the t-th iteration can be 

denoted as Q'{t) = (Sf (t), • • • , Qf„ ' • ■ , Q;l W) 
with quantization rate vector (at the t-th iteration) 
L"(i) ==: (Lf(t), ■ • • Similarly, the system 

vector TVQ at the t-th iteration can be denoted as 

Q-{t) = (QUt),--- ,Qlit)r-- ,QUt)) with quan- 
tization rate vector (at the t-th iteration) 'L'"{t) = 
{Ll{t), ■ ■ ■ ,Ll^{t)). Using the resuh (c) of Lemma |2l 
the TVCOQ design, which minimize E^^^^^{T) under 
total quantization rate constraint over a horizon of T 
iterations, is equivalent to the following: 
Problem 2 (TVCOQ Design Problem): 



mm a 

{Q=(t):0<t<T} 
)r{Q"(t):0<t<T} 

f-1 n 



T-1 



«-1e,||bTock 



(25) 



E E = TL,L^^{t) e Z+(Vm,i)SQ 



t=0 m=l 



(26) 
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T-1 K 

orY^Y. ^kit) = TL, Ll{t) G Z+(Vfc, ^VQ (27) 

t=0 fc=l 

By introducing additional auxiliary variables {L{t) : 
Q < t < T — 1], where L{t) can be interpreted as 
the per-stage sum quantization rate, and applying primal 
decomposition techniques, we can decompose Problem 
|2] into subproblems (per-stage TICOQ design Qf {t) or 
Q"(t) {Q<t<f- 1)), which are given by 

Problem 3: (TVCOQ Subproblems: Per-stage TICOQ 
Design Problem) 

min ||ei(L(<))^„,, (28) 

S-'(t)oi-Q"(t) 
n 

s.t. ^L,^„(i) = L(t),L^(t)eZ+(Vm)SQ (29) 



m— 1 
K 



or ^L],:(i)=i(t),i]J(t)GZ+(Vfc)VQ 



(30) 



fc=i 



and the master problem (per-stage sum quantization rate 
{L{t) : < t < T — 1} allocation among stages), which 
is given by 

Problem 4: (TVCOQ Master Problem: Sum Quanti- 
zation Rate Allocation Problem) 

f-i 



min a^-^Y a-'\\e*JL{t))\ 



T-1 



I block (31) 

S.t. Y = e z+(0 < t < T - 1) (32) 

where |ie*(i(i))|| 

block '^he optimal value of the t-th 
subproblem in Problem[3]for given L{t) (0 < t < 1). 
Given the per-stage sum quantization rate L{i), each Per- 
stage TICOQ Design Problem in Problem |3] is the same 
as the TICOQ design problem in Section IIVI and hence, 
the approaches in Section |IV] can be applied to solve 
Problem |3] for given L{t) {Q < t < T - 1), including 
both SQ case and VQ case. In the following, we shall 
mainly discuss the Sum Quantization Rate {L{t) : < 
t < T — 1} Allocation in Problem]?] and analyze the 
tradeoff between convergence error and message passing 
overhead for the TVCOQ design. 

A. Time Varying Convergence-Optimal Scalar and Vec- 
tor Quantizers 

Similar to the TICOQ in Problem JU each TVCOQ 
subproblem in Problem ]3] is a Nonlinear Integer Pro- 
gramming (NLIP) problem. Brute-force solution to the 
TVCOQ master problem in Problem ]4] requires exhaus- 
tive search, which is not acceptable. Therefore, we first 
apply continuous relaxation to the subproblems in Prob- 
lem ]3] to obtain the closed-form expression He^ Hy^^^j^ of 



the t-th subproblem. Based on the closed-form HejUyock' 
the master problem in Problem ]4] becomes tractable, the 
solution of which is summarized in the following lemma. 

Theorem 6: (Solution to TVCOQ Master Problem for 
SQ and VQ) For any given T, assume L > L' — 
n^log^a (L e Z+). Let L*{t) = n\og^{^^^), 
where fi > is the LM of the constraint ( l32b chosen to 
satisfy the constraint Y^JSq r7,log2( ° = TL. The 

optimal integer solution to the TVCOQ Master Problem 
in Problem ]4] is given by 



L*{[t]) 



\L*{m, 



^ft<Ej=oiL*{t)-lL*{t)\) 
otherwise 

(33) 



log2 a (L G Z+), where L' is given by 



Proof: Please refer to Appendix G for the Proof. ■ 
Given the per-stage sum quantization rate {L*{t) : 
< t < T — 1} allocation obtained by Theorem ]6] 
the TVCOQ Subproblems in Problem ]3] (similar to the 
TICOQ design problem in Section llVb can be easily 
solved by Theorem ]3] ]4] and ]5] 

B. Tradeoff between Convergence Error and Message 
Passing Overhead 

In this subsection, we shall quantify the tradeoff 
between the error of the algorithm trajectory and the 
message passing overhead using TVCOQ. The following 
lemma summarizes the tradeoff results. 

Lemma 6: (Performance Tradeoff of the Scalar and 
Vector TVCOQ) For any given T, the convergence error 
bound (at the T-th iteration) of the scalar and vector 
TVCOQ is given by i^bTock(^') = fa^O{2-i) for 
L>L' 
M- 

Proof: Please refer to Appendix G for the proof. ■ 

Remark 7: As L —> oo, E^^^^{T) -> 0, which reduces 
to the conventional convergence results with perfect 
message passing. On the other hand, for any fixed L, as 
T ^ cx), we have E^^^^if) 0. Hence, using TVCOQ, 
one could achieve asymptotically zero convergence error 
even with finite L. ■ 

VI. Simulation Results and Discussions 

In this section, we shall evaluate the performance of 
the proposed time invariant convergence-optimal quan- 
tizer (TICOQ) and time varying convergence-optimal 
quantizer (TVCOQ) for the contracting iterations by 
simulations. We consider distributed precoding updates 
in MIMO interference game with K transmitter and 
receiver pairs where the transmit convariance matrix 
of the k-th transmitter is iteratively updated (and quan- 
tized) according to (T5[ . In the simulation, we consider 
K = 2, 4, 8 noncooperative transmitter-receiver pairs 
with = 2, 4 transmit/receive antenna. The distance 
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Per-stage Sum Quantization Rate (L) 

(a) Sum throughput 




Per-stage Sum Quantization Rate (L) 



(b) Convergence error 

Fig. 2. Sum throughput/convergence error versus per-stage sum 
quantization rate L (bits) of 2 pairs MIMO interference game with 
2 transmit and receive antennas, d\\ = d22 = 100 m, di2 = 200 
m, d2i = 500 m, path loss exponent 7 = 3.5, and transmit power of 
Pi = P2 = 10 dBm. The total number of iterations is f = 4 and 
the per-stage quantization rate per antenna is j^^j^'a = In (b), the 
"*", "o", etc represent the simulation results of the proposed TICOQ 
and TVCOQ, while the dashed line represents the analytical expression 

0(2"^) (TICOQ) and faT^C'(2"i) at fixed f = 4 (TVCOQ). 

from the fc-th transmitter to the j-th receiver is denoted 
as dkj- The bandwidth is 10 MHZ. The pathloss exponent 
is 7 = 3.5. Each element of the small scale fading 
channel matrix is C7V(0, 1) distributed. We compare the 
performance with two reference baselines. Baseline 1 
(BLl) refers to the case with perfect message passing 
Is). Baseline 2 (BL2) refers to the case with uniform 
scalar quantizer. 

A. Performance of the TICOQ 

Fig. |2] (a) and Fig. |2] (b) illustrate the sum through- 
put and convergence error (w.r.t. the weighted block- 
maximum norm || • W^x^^^f) versus the per-stage sum 
quantization rate of 2 pairs MIMO interference game 
under fixed number of iterations. The sum throughput 
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Instantaneous Iteration Index (t) 



(a) Sum throughput 
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Instantaneous Iteration Index (t) 



(b) Convergence error 

Fig. 3. Sum throughput/convergence error versus instantaneous 
iteration index t of 4 pairs MIMO interference game with 4 transmit 
and receive antennas, da = 100 m, dij = 200 m (i < j), dij = 500 
™ > j)> path loss exponent 7 = 3.5, and transmit power of 
Pi = P2 = 5 dBm. The per-stage sum quantization rate L = 64 bits 
(i.e. the per-stage quantization rate per antenna is ^ ^^^2 ~ 53 ~ ^ 
bit), and the total number of iterations T = 7. 



of all the schemes increases as L decreases due to the 
decreasing convergence error. It can be observed that 
the proposed scalar and vector TICOQ have significant 
performance gain in sum throughput and convergence 
error compared with the commonly used uniform scalar 
quantizer In addition, the vector TICOQ outperforms the 
scalar TICOQ in convergence performance at the cost of 
the higher encoding and decoding complexity. 

Fig. |3] and Fig. |4] show the sum throughput and 
convergence error versus the instantaneous iteration time 
index of MIMO interference game under a fixed per- 
stage sum quantization rate and total number of iterations 
with different K and N. It can be seen that in all cases, 
the proposed scalar and vector TICOQ have significant 
performance gain in sum throughput and convergence 
error compared with the commonly used uniform scalar 
qunatizer 
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(a) Sum throughput 




1 2 3 4 5 6 

Instantaneous Iteration Index (t) 



(b) Convergence eiTor 

Fig. 4. Sum throughput/convergence error versus instantaneous 
iteration index t of 8 pairs MIMO interference game with 2 transmit 
and receive antennas, da = 100 m, dij = 200 m {i < j), dij = 500 
™ (* > P^th loss exponent 7 = 3.5, and transmit power of 
Pi = P2 = 5 dBm. The per-stage sum quantization rate L = 32 bits 
(i.e. the per-stage quantization rate per antenna is ^ ^^ i ^ ~ ~ ^ 
bit), and the total number of iterations f = 6. 

B. Performance of the TVCOQ 

From Fig. |2] (a) and Fig. [2] (b), we can observe that 
the proposed TVCOQ has significant performance gain 
in sum throughput and convergence error compared with 
commonly used uniform scalar qunatizer and TICOQ. 
For example, the sum throughput of the TVCOQ is very 
close to that with perfect message passing. Fig. [3]and|4] 
illustrate the transient performance of the TVCOQ versus 
iteration index t. We observe that the performance of 
the TVCOQ improves as t increases. This is because 
the TVCOQ optimizes the quantization rate allocation 
over both the node domain and the time domain (over a 
horizon of T iterations). 

C. Tradeoff between Convergence Error and Message 
Passing Overhead 

Fig.|2](b) illustrates the tradeoff between convergence 
error and message passing overhead (in terms of L at 



11.5 
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(a) Sum throughput 
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(b) Convergence error 

Fig. 5. Sum throughput/convergence error versus the total number of 
iterations T of 2 pairs MIMO interference game with 2 transmit and 
receive antennas, dn = d22 = 100 m, di2 = 200 m, d2i = 500 m, 
path loss exponent 7 = 3.5, and transmit power of Pi = P2 = 10 
dBm. The per-stage sum quantization rate L = 8 bits (i.e. the per-stage 
quantization rate per antenna is j^^i^-i ~ ^ ~ ^ t''^- '^he "*", 

"o", etc represent the simulation results of the proposed TICOQ and 
TVCOQ, while the dashed line represents the analytical expression 

0(2" (TICOQ) and f 0^^0(2' with T > 1 (TVCOQ) at 
fixed L = 8 bits. 

fixed total number of iterations T). As the message 
passing overhead L increases, the convergence error of 
all the proposed quantization schemes approaches to 
with order 0{2^~) under fixed T, which verified the 
results in Lemma [5] and |6] Similarly, the sum throughput 
of all the schemes increases as L decreases due to the 
decreasing convergence error as shown in Fig. |2] (a). 

Fig. |5] (b) shows the tradeoff between convergence 
error and message passing overhead (in terms of T at 
fixed L). As the total number of iterations increases, 
the convergence error of TVCOQ decreases, while the 
convergence error of the TICOQ and the uniform quan- 
tizer fail to decrease. It is because that TICOQ has 
steady state convergence error floor for any finite L (i.e. 
-^wock(oo) = T^C'(2-^) > shown in Lemma IS, 
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while the convergence error of TVCOQ goes to as T 
goes_to infinity (i.e. E^^,i,if) = fa^O{2-^) 
as T — > cxD shown in Lemma |6]l. Similarly, as the 
total number of iterations increases, the sum throughput 
performance of the TVCOQ improves but this is not the 
case for TICOQ and the Baseline 2 as shown in Fig. |5] 
(a). 

VII. Summary 

In this paper, we study the convergence behavior 
of general iterative function evaluation algorithms with 
quantized message passing. We first obtain closed-form 
expressions of the convergence performance under quan- 
tized message passing among distributed nodes. To min- 
imize the effect of the quantization error on the con- 
vergence, we propose the time invariant convergence- 
optimal quantizer (TICOQ) and the time varying 
convergence-optimal quantizer (TVCOQ). We found that 
the convergence error scales with the number of bits for 
quantized message passing in the order of y^O(2~" ) 

and Ta^O(2^*) for TICOQ and TVCOQ respec- 
tively. Finally, we illustrate using MIMO interference 
game as example that the proposed designs achieve 
significant gain in the convergence performance. 

Appendix 

Appendix A: Proof of Lemma[T] 

First, we prove conclusion (a). By the update equation 
in (O, the triangle inequality of norm and the property 
of the contraction mapping, we have ||x(i) — x*|| = 
|[T(x(t-l))+e(t-l)-x*|| < ||T(x(t-l))- 
xll + ||e(t - 1)11 < a||x(t - 1) - x*|| + ||e(t - 1)|| = 
a||T(x(< - 2)) + e{t - 2) - x*|| + ||e(t - 1)|| < • ■ • < 
a* ||x(0) - x* II + E{t), where E{t) = Y!i=i \W - 



-r ||e(r)" — ^ \ " 'I 



E 



i'=0' 



(1) is obtained by denoting I' ^ t — l, and (2) is obtained 
by denoting 1 = 1'. (b) is trivial. Finally, we prove 
conclusion (c). Since ||e(i)|| < ||e|| Vt, we have E{t) = 



EU«'-'l|e(i-OII < E{t) ^ ELi 

i^||e|| and ^(oo) ^ \iTat^^E{t) = j^. Given the 
limiting error bound E{oo), we know that 3T s.t. Vt > 
T, x(t) e §. If X = Q(T(x)) Vx e §, then x(i) ^ 
x(cxd) e S. Thus, we obtain the sufficient condition 
for convergence. On the other hand, if x 7^ Q(T(x)) 
Vx G S, x(t) will not converge, but jumps among (at 
least two) points in §. Thus, we obtain the necessary 
condition for convergence. 

Appendix B: Proof of Lemma|2] 

Jacobi scheme in ( fT6l ) shares the similar form as (fT4l l. 
Therefore, the proof of Jacobi scheme is the same as 



that in Appendix A, except based on weighted block- 
maximum norm. 

Next, we prove the convergence for the Gauss-Seidel 
scheme under quantized message passing. Let x'^ = x 
for fc = 1 and i'^ = (Si(x) + ei,-- - ,Sa;_i(x) + 
e/j_i,Xfe, - - ,xx) for 2 < fc < A'. By the definition 
of weighted block-maximum norm and the property of 
block-contraction T, we have 



|Sfc(x) 



■n.\\k 



|Tfe(x''0-Tfc(x*)|U 



<||T(i^)-T(x*)|| 
- <a||x-x*||-,„ 



block 



< alli*^ -X* 



; I maxj< 



= a max maxj^k 



||Sj(x)+ej-x;| 
Xj-x*|| 



I block 
fc = 1 



2<k<K 
(34) 



When fc = 2, by ( l34l i. we have 

I|S2(X)-X*||2 



W2 



<a max ■ 



<a max • 



|Si(x)+ei-x|||i 



Wi 

|Si(x)-xJ||i 



, max ■ 

J>2 



|ei 1 



Wl 



<a||x-x*||^„,k- 



a e 



Wl 
by iteration 



max ■ 

i>2 



block 



|Sfc(x)-x 



kWk 



Wk 



< a||x-x*||^„,k 



fe-i 

E 

1=1 



"'l|e||bTock,Vfc 



|S(x)-x*||-,k<"l|x-x*||-,, 
a(l -a^'-i) 



^||x(i)-x*^„,, 
= ||S(x(t- 1)) +e(f- 1) -X 

<a*||x(0)-X*||S'oek+i?bToclcW 



* II w 
block 



where £;-,,(t) 



bloclcV 



Ta^"*~'Eto«-'l|e(OllMock- 
Since we have shown \\x{t) ~ x*\\uock — 0^*11^(0) ~ 
X* 1 1 block + -^bk)ck(^)' which is the same as the conclusion 
in Lemma [T] (a) except for the different norm || • W'^^^.j^ 
and the extra scalar > 1 (indicating the additional 

error due to the incremental nature of the Gauss-Seidel 
update) in E^g^^^{t), we can follow the similar steps in 
Appendix A to obtain the conclusion for Gauss-Seidel 
scheme. 

Appendix C: Proof of Lemma[3] 

y u is monotone (ab- 



First, we shall show that 



solute). Denote |x| = (|xi|,-- - ,|x„|). We say that 
|x| < |y| if l^ml < |ym|Vm. Due to the monotonicity 

of II • life (Vfc-), we have |x| < |y| ^ |xfe| < |yfe| (Vfc) ^ 
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llxfcllfe < llYfellfc (Vfc) maxfc ^ < max^ ^ ^ 
ll^llwock < llyllwock- Thus, II . II-,, is monotone (abso- 
lute). Next, we shall show that each coordinate scalar 
TICOQ is uniform quantizer. Given any L'' s.t. ( |20] | is 
satisfied, by the monotonicity of || • Hyo^k' easily 
prove minQ.(L3) ||e||^„,|^ <^ mine=^(L^) e„(Vm). In 
other words, given any Lf^, the coordinate scalar TICOQ 
Qm{Lf^) should minimize the worst-case error |e„i| for 
the TO-th coordinate of the input vector Since the uniform 
quantizer minimizes the worst-case error regardless of 
the shape of the input pdf 1201 . each coordinate scalar 
TICOQ Qf* is a uniform quantizer 

Appendix D: Proof of Theorem[3]and Theorem 

s 

When II ■ life is weighted maximum norm, the ob- 



jective function becomes ||e| 
— = max„ 



maxfe 

max^ 
miriL' 



maxfe ■ 



C™2-^". Therefore, we have mino^ 
(max„i Cm2~^m). By continuous relaxation and 
equivalent transformation of minimax problems |25l, the 
minimax problem in Problem [T] is equivalent to the 
following problem (under continuous relaxation), which 
is in epigraph form with optimization variables {Lf^}, r: 

(P") : mill T 

S.t. Cm2'^'"' <t{1 <m< n) (35) 

n 

Y,L'^ = L,L:^>0{l<m<n) 

ni—1 

(36) 

{P") is a convex optimization problem. It can be 
easily shown that the Slater's condition holds. There- 
fore, we shall get the optimal solution to the re- 
laxed problem through KKT conditions. The La- 
grangian of (P'^) is given by £*(L^, r, A*, 1/^, /i*) = 

+ Sm=l-^m(Cm2 - t) - J2rn=l'^m^m + 

{J2m=i -^m ~ -^)' where A* , , fi^ are the Lagrangian 
multipliers (LM). L'', r. A'' , i/** , /i* are optimal iff 
they satisfy the following KKT conditions: (a) pri- 
mal constraints: (I36]l.(l35li: (b) dual constraints: A* >z 
0, y 0; (c) complementary slackness: Af„(C„i2^^™ — 
r) - (Vm), - (Vm); (d) |g- = 

-ln(2)A^C™2-£™ - < + /i^ - (Vm), ^ = 
1 ^ = 0- Thus, Vm, we have 

ifi^^^?l^>l:Af„ = ^,.^ = 0, 



= l0g2( 



ln(2)r 



ln(2)A^ 



if ^^^^^^^^ < 1 : = 0, = M^ i^t = 0, > 



where LMs {A^},/i* are chosen to satisfy 
E:=i(log2 ^ L and 1 - = 0. 



Substitute 



into if 



we have 



~ l"(2)Af,. 

— (log2 ^)^, where r is chosen to satisfy 
X]m=i(i'^S2 — )^ = i. Furthermore, substituting the 
relaxed solution Lf* into the transformed problem 
(P"), the optimal value of (P") is given by r 
and this is also the optimal value 116*11^^^^,,^ of the 
original optimization Problem [T] (under continuous 
relaxation) due to the equivalence of the epigraph 
transformation. Next, we are trying to prove that the 
rounding strategy in (|22]) in Theorem [5] is the optimal 
integer solution of Problem [T] Suppose we round 
to Lif,:J and let S^^ - L» - [^^J- Denote 
h — (1 2^Li,"J — n o^iKl-^D {eh — (7 
if Cm < T and b„i = 2^^t otherwise. The value 
of the objective function in Problem [T] is max™ hm- 
Reducing L^l by TV + (5,"„ (iV > 1, iV e Z+) 
Vm for integer solution will lead to the value of 
objective function greater than max^ hm. On the other 
hand, since the optimal value of the original integer 
programming problem is greater than the optimal 
value under continuous relaxation, increasing 
by - (5^ (A^ > 1, iV e Z+) Vm will not help 
further reducing the value of the objective function. 
Therefore, the optimal integer solution {Pm} satisfies 
— 1 < Pm — Pm — and the rounding strategy in 
is the optimal integer solution of Problem [T] 
When II • llfc is Pj, norm, the objective function becomes 



llefclli 



l|e|lbiock = maxfc^^ = max^ - 

maxfc(EmeA^fc <^™2"P^")p. ^ There- 

fore, we have miiiQ^ J|e||^„,|, = 
minL= (maxfc(X;„g^^ C',„2"P^"-)p). Using similar 
continuous relaxation and equivalent transformation 
of minimax problems [25], the minimax problem in 
Problem [T] is equivalent (under continuous relaxation) to 
the following problem, which is in epigraph form with 
optimization variables {Pf„},r: 

(Q*) : min t 

S.t. ^ C„i2"P^'" < T (1 < fc < PT) (37) 

meA^fc 

constraint ind 



(Q*) is a convex optimization problem. Using similar 
argument as in the weighted maximum norm case, the 
Lagrangian of (Q*) is given by £"(£1'', r, A", 1/*, /i*) = 

+ Afc (SmeTMfc C'm2~P-'""-T)-X]m=l ^m^m^^ 

Ai'(Em=i-^m - where \\v\ii^ are the LMs. 
Using standard KKT conditions, the optimal solution 



of m is p^* = Mog: 
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1 and 



i til 

Vpln(2 



— i-TTTYT^, then we have 
■ V 1), where {rfc} and 



jmeMk ^"i^ ~ Z-^meMk Vpln(2)A| 

Cm) — 'T > 0, X V a = max{a;,a}, x A a = 

min{a;, a}, where the LMs {A^.}, /i'* are chosen to satisfy 

and Ef=i -^1 = 1- Let Tk = 

T S* _ 1 ( Cm 

T are constants related to the LMs {A^}, /i* of the 
constraints (|36]|.(|37]| in problem (Q*). They are chosen 
to satisfy T,k=iT,meM, ^^og^i^^ V 1) = L and 
^ ( EmeM, (C^m A Tfe) - r) = (Vfc). Finally, substitut- 
ing Lf* into the transformed problem (Q*), the optimal 
value of (Q*) is given by rp and this is also the optimal 
value ||e* of the original optimization Problem [T] 
(under continuous relaxation) due to the equivalence of 
the epigraph transformation. 

Appendix E: Proof of Lemma|4]and|5] 



Since llxfelU < WykWk (Vfc) => ||x||b^,k < 
lyllbiock' given any s.t. (l2Tl i is satisfied, we have 
mins"(L") M^iock ^ minQj:(L;;) l|efc||fc (Vfc). The type 
of each component vector TICOQ Q^* is uniquely 
determined by the norm || ■ ||fe on R"*. Furthermore, it 
is easy to prove that should be a lattice quantizer to 
minimize the worst-case error ||efc||fc. We shall discuss 
the two cases for L2 norm and weighted maximum norm 
separately. 

• II • Ik- (1 < fc < K) is L2 norm: The cov- 
ering problem asks for the thinnest covering of 
R"'" dimensional space with overlapping spheres, 
i.e. minimizes covering radius (circumradius of the 



Voronoi cell) pk 



\\ek\\k 



(E 



112411 . Therefore, each component vector TICOQ 
Q1* minimizing the worst-case error is the thinnest 
lattice for the covering problem. 
• II • life (1 ^ fc < K) is weighted maximum 
norm: Given any L^, we have minQjj(^ij) ll^/cH^ = 

minQ^(ijj) maxmeMk ^f^- It can be easily shown 
that each face of the Voronoi cell for the weighted 
maximum norm is (n^ — 1)- dimensional hyperplane 
parallel to a coordinate axis in the Uk dimensional 
space. Therefore, it is equivalent to the scalar quan- 
tization of each coordinate Xm of the input block 
component with different scalar quantizers, i.e. 

Next, we shall show the optimal solution for the 
dual lattice quantizer. For A'^^, the covering radius 

is Rn^ ~ \/ i2(rtfc+ lf' '•^^ volume of the fundamen- 

The volume of bounded region 

Therefore, the worst 



tal region is 

Xk is v,^ = n 



rifc + l 



LI is given by ||efe|lfe = 



"fc("fc+2) . 



The covering ra- 



dius measured by Lp norm (p > 2) can be proved to 

be less than Rn,. — \J ^2(nli^ ' which is the covering 
radius measured by L2 norm. Thus, the worst case 
error is also less than ||efc||fc given above. Therefore, in 
general, we can apply A* quantizer for VQ case when 
II ■ life is Lp norm [p > 2) and consider TICOQ design 
for VQ cased based on A* quantizer 

Problem [1] for A* quantizer is equivalent to 

miriL" ( maxfc £'fc2 "fc )s.f:.(l2Tli. Similar to Appendix 
D, the optimal solution under continuous relaxation is 
LJJ* = rife (log2 -^)^, where r is a constant related to the 
LM and is chosen to satisfy J2'm=i '^fe(log2 — )^ = L. 
For rii = • • ■ = hk, we can use similar argument as in 
Appendix D to show that the rounding method in ( l24l i is 
optimal integer solution to Problem [T] (VQ case). 

Appendix F: Proof of Lemma[5] 
We try to find L' in the following three cases s.t. when 
L > L', we have Cm > t (Vm), Cm > X^fcLi G 
Mk] (Vm) and Dk > r (Vfc), separately (to obtain the 
closed-form optimal value ||e* ||J5[ock)- Specifically, we 
have 

• SQ (II • life is weighted maximum norm) in Theorem 
[3] (same as VQ (|| • 1^, is weighted maximum 
norm)): Cm > r (Vm) ^ = (loga %-)+ - 
log2^ (Vm). Since loga ^ = L ^ 

llellbTock = r = 2^^™^ii°g^^'"-^ = Oi2-i), 
we have Cm > t (Vto) <^ mirim Cm > 

2^i:;u.iog,c„-#^ ^ i > E:=iiog2C™- 

nlog2(min„ Cm) = L'. 



SQ (II 

Cr 
Li 



\k IS 

Cm > J2k=l '^k'l[m 

pl0g2(^^^^I[„g^ 

p^°S2ij2j^^^rS[mGM 



Lp norm) in Theorem H) 



f) 



(7 2~P^" 



= nkTk 
= L 



(Vm) 

V 1) = 
(Vm). Since 

le' 



r 



1=1 

I w 

I block 



case error of the vector TICOQ with quantization rate cases we are interested in. 



p Em=l l0g2 -? 

r-. = 2i(^^;"-ii°s^(^-)-^) = 0(2-#). 
Similarly, we have L' = J2m=i log2 (C*™) - 
nlog2 (min„ Cm)- 
. YQ (A; quantizer) in Theorem [5] > r (Vfc) <^ 
= nfc(log2^)+ = nfelog2^ (Vfc). Since 
Eti"fclog2T" = L ^ lie* II-,, = r = 
27ri:Li"fci°g2-Dfc-i ^ 0(2-i). Similarly, we 
have V = I]f=i log2 Dk - ii'log2(minfc Dk). 

"Note that boundary effect is ignored liere. Tlie performance loss 
is negligible when L is large, which is easily satisfied in most of the 
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Appendix G: Proof of Theorem[6]and Lemma[6] 

First, we shall find the requirement for L s.t. each 
subproblem (under continuous relaxation) has closed- 
form \\el (L{t)^ llbiock obtain the closed-form objective 
function of the TVCOQ master problem). By Appendix 
F, to obtain closed-form He^ (L(i)) 11^^,^,]^, we require 
Lit) > i' (0 < i < T - 1). Under this assumption, 
we have \\eUL(t))\\^^^,. = r^2-^, where 



V= < 



=t (iW)llHock 

2i ^k=i "fc i°g2 Dk 



WM norm (SQ) 
Lp norm (SQ) 
dual lattice (VQ) 



(38) 



Therefore, the objective function in Problem |4] becomes 



rja 



T-l 

t=0_ 



a 



L(t) 



and Problem H] is equivalent to 

1 — f -L(t) J . 

a 2 " s.t.(|32t. By continuous re- 
laxation and standard convex optimization techniques 
(similar to Appendix D), we have the optimal solution 
(under continuous relaxation) L*{t) — nlog2( " 

Since L{t) > L' (Vt), L*{t) = nlog2(^''"^^ 

^^ELV (log2(ln2) 



71 fl 



log2('^M)) = nnog2(ln2) - 
ntlog^inii) ^ TL ^ L{t) 



(T-l)T 
2_ 



log2(a) 
"-^log2(a) 



nnog2(a) + L. Since L{t) increases with t, to satisfy 
L{t) > V (it), we require i(0) = n2^1og2(Q:) + 
L > L' ^ L > L' — "-^-j^ log2 a. Therefore, when 
L > L'- 



, T-l 



log2 a, we have L*{t) = nlog2(^ 



ln2\ 



' In 2 



Tn2 ■ 



and «-ie*(L*(i))||S'„,, = r; • a" ^ 
Similar to Appendix D, the rounding policy in ( |33] ) can 
be shown to be optimal. 

Next, we shall analyze the tradeoff between conver- 
gence error and message passing overhead for L > 



that L{t) 



L' — n-^-^ log2 a (L G Z+). Since it has been shown 



770; 
a* ■ 
Ta' 



, T-l 
2 

-1 - -t 



n-^-TT-^ log2(Q;) — nZlog2(Q;) + L, we have 



-0(2- 



a -2-- 
= fa- 



= rja 
-0(2- 



T-l _t/ -II;^ 
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